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ERRATUM 


The  second  reference  on  page  6l  should  be 


Swets,  J.  A.,  Green,  D.  M.}  Getty,  D.  .T . , and  Swets,  J.  B. 
Identification  and  scaling  of  complex  visual  patterns.  (Technical 
Report  3536).  Cambridge,  Massachusetts:  3olt  Beranek  and  Newman 
Inc.,  November,  1977. 
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& relative  loci  of  stimuli  in  that  space,  and  a probabilistic 
decision  model  based  on  weighted  interstimulus  distances  was 
developed  to  predict  the  confusion  matrices  in  various  identi- 
fication tasks.  The  high  accuracy  of  the  predictions  supplies 
a strong  validation  of  the  use  of  multidimensional  scaling 
procedures  to  reveal  perceptual  structure.  In  demonstrating  the 
ability  of  that  structure  to  account  for  behavior  In  an 
independent  task.  From  the  other  point  of  view,  the  empirical 
success  of  this  approach  suggests  a relatively  simple  and 
practical  means  of  predicting,  and  possibly  enhancing,  identi- 
fication performance  for  a given  ensemble  of  visual  or  auditory 
stimuli. 
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Figure  Capt ions 


Figure  1.  Fxample  of  a stimulus  configuration  for  which  an 
asymmetric  confusion  matrix  would  he  obtained 
( see  text). 

Figure  2.  Long-term  spectra  of  eight  underwater  sounds 
(From  Howard,  1976). 

Figure  3.  Visual  representations  of  the  eight  underwater 
sounds,  photographed  from  the  display  monitor. 

Figure  4.  8x8  Experiment:  Probability  of  a confusion 

error  for  each  of  the  eight  stimuli,  for  each 
observer . 


Figure  5 , 


Figure  6 , 


Figure 


Figure  8 . 


Figure  9 


8x8  Experiment:  Distribution  of  response 
probability  for  each  stimulus,  for  each  observer 
Obtained  distributions  are  given  by  solid  lines 
and  filled  circles;  distributions  predicted  by 
the  model  are  given  by  dashed  lines  and 
open  circles . 

8x4  Experiment,  Condition  1:  Distribution  of 
response  probability  for  each  stimulus,  for  each 
observer.  Obtained  distributions  are  given  by 
solid  lines  and  filled  circles;  predicted 
distributions  are  given  by  dashed  lines  and 
open  c i rc 1 es  . 

8x4  Experiment,  Condition  ?:  Distribution  of 
response  probability  for  each  stimulus,  for  each 
observer.  Obtained  distributions  are  given  by 
solid  lines  and  filled  circles;  predicted 
distributions  are  given  by  dashed  lines  and 
open  circles. 


Experiment,  Condition 


Distribution  of 


response  probability  for  each  stimulus,  for  each 
observer.  Obtained  distributions  are  given  by 
solid  lines  and  filled  circles;  predicted 
d i s t r i but  ions  are  given  by  dashed  lines  and 
open  circles. 

Estimated  salience  weights  on  the  four  dimensions 
for  each  observer,  for  each  condition  in  the 
8x8  and  8x4  experiments  (solid  lines). 

The  pattern  of  weights  that  maximizes  probability 
of  correct  identification  is  given  by  the 
dashed  line  for  each  condition. 
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A reasonably  complete  account  of  the  process  by  which 
humans  are  able  to  identify  complex  auditory  or  visual 
stimuli  must  address  two  issues:  (1)  the  nature  of  the 
psychological  representation  of  complex  stimuli,  and  (2)  the 
nature  of  the  decision  processes  that  act  upon  the  internal 
representat ions  to  yield  an  identification  response.  While 
there  has  been  substantial  research  directed  at  the 
representation  and  decision  processes  separately,  there  has 
been  much  less  effort  directed  towards  understanding  the 
integration  of  these  processes  in  complex-stimulus 
identification.  Our  major  concern  in  this  paper  is  with  the 
relationship  between  the  perceptual  representation  of 
complex  stimuli  and  identification  performance.  The 
question  is,  given  specific  assumptions  about  the  structure 
of  the  perceptual  space,  how  well  can  we  account  for  the 
pattern  of  responses  observed  in  an  identification  task? 

Our  approach  to  the  problem  involves  two  parts:  (1)  the 
derivation  of  a multidimensional  perceptual  space  for  a set 
of  complex  stimuli  from  the  application  of  a 
multidimensional  scaling  (MDS)  procedure  to  judgments  of 
stimulus  similarity,  and  (2)  the  use  of  a probabilistic 
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decision 

model  to 

predict  the  matrix 

of 

identification 

con  fus i ons 

from  the 

geometric  structure 

0 f 

the  derived 

perceptua 1 

space . 

We  are  interested  in 

the 

val idation  of 

the  MPS  procedure  that  would  be  supplied  by  a demonstration 
that  the  MPS-derived  perceptual  space  can  be  used  to  predict 
behavior  in  an  independent  task.  And  we  are  interested  in 
the  possibility  that  the  MPS  procedure  can  be  a substantial 
aid  in  understanding,  and  predicting  behavior  in,  the 
fundamental  task  of  stimulus  identification. 

We  discuss  first  the  rationale  for,  and  assumptions 
made  in,  inferring  a psychological  space  using  MPS 
techniques.  Then  in  the  next  section  we  describe  our 
decision  model  for  predicting  identification  confusions. 

The  Multidimensional  Perceptual  Space 

Much  of  the  work  on  the  representat ion  of  complex 
auditory  or  visual  stimuli  suggests  that  perception  is  based 
on  an  analysis  of  the  patterns  along  a number  of 
psychological  dimensions  or  features.  In 
information-processing  models,  this  processing  is  often 
referred  to  as  "feature  extraction"  and  is  thought  to 
reflect  a selective  reduction  of  information  whereby 
perceptually  important  features  are  extracted  from  the 
pattern  while  other  information  is  lost. 
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We  may  conceive  of  these  dimensions  as  forming  a 
multidimensional  perceptual  space  in  which  each  stimulus  is 
represented  as  a point.  This  space,  of  course,  is  not 
directly  observable.  Both  the  set  of  dimensions  comprising 
the  space,  and  the  loci  of  the  stimuli  within  the  space, 
must  be  inferred  by  indirect  methods.  Extensive  development 
in  recent  years  has  led  to  the  emergence  of  multidimensional 
scaling  as  an  important  method  for  deriving  a representat ion 
of  the  perceptual  space  (e.g.,  Shepard,  Nerlove,  and  Romney, 
1972;  Romney,  Shepard,  and  Nerlove,  1972).  MDS  procedures 
are  designed  to  decompose  a matrix  of  pair-wise  similarity 
judgments  on  a set  of  complex  stimuli  into  a metric  space  of 
some  (investigator-specified)  number  of  orthogonal 
dimensions.  Each  stimulus  is  defined  as  a point  in  the 
space  such  that,  ideally,  the  distances  between  pairs  of 
stimuli  in  the  space  are  monoton ical 1 y related  (inversely) 
to  the  degrees  of  judged  similarity  of  the  pairs. 


The  set  of  abstracted  dimensions,  and  the  relative  loci 
of  the  stimuli  within  the  space,  may  be  interpreted  to 
reflect  the  structure  of  the  psychological  space.  This 
inter pretation  involves  several  assumptions.  Most  MDS 
procedures  assume  that  the  measure  of  the  underlying 
psychological  space  is  a member  of  the  family  of  power 
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metrics.  (11  This  family  includes  the  Fuel  id  i an  and 
city-block  metrics.  The  power  metrics  carry  with  them  three 
important  properties:  (11  decomposabil ity , (2) 

i nterd imens i onal  additivity,  and  (11  intrad imensional 
sub t r ac t i v 1 1 y (Tversky  and  Krant. 7. , 19701.  Decomposabil  ity 
means  simply  that  the  distance  between  any  two  points  in  the 
space  is  a function  of  dimension-wise  contributions. 
In  t.erd  imens  ional  additivity  asserts  that  distance  is  a 
function  of  the  sum  of  the  dimension-wise  contributions. 
Intrad imensional  sub  tract iv  i ty  asserts  that  each 

dimension-wise  contribution  is  the  absolute  value  of  the 
difference  between  the  two  points  on  that  dimension.  An 
intuitive  implication  of  these  properties  is  that,  for  a 
given  pair  of  stimuli,  the  contribution  to  distance  made  by 
the  values  of  the  two  stimuli  on  one  dimension  is 
independent  of  the  values  on  all  other  dimensions. 

Having  obtained  an  abstract  multidimensional  solution, 
an  investigator  may  attempt  to  relate  the  derived 

HI  Power  metrics  are  the  class  of  metrics  such  that  the 
distance  between  the  points  x = (xp...,x  1 and 
y = (y,  , . . . , yt1>  is  given  by 

" u rl1  r 

d ( x , y 1 » h | x i - ,v  , | 

_ 1 - 1 

for  r > t. 


" 'V 


Koport  No.  \’10  Ho  1 t horanok  ami  Nowni.m  Ino. 

psyoho  l og  i on  1 .1  linens  i ons  to  t ho  known  ptiy.sio.il  si  mot  lire  of 

t ho  stimuli.  Suooess  in  Identifying  the  psyotu'ptiy  s l o.i  1 
f uno  t Ions  relating  psyoho l og i on l to  pliysio.il  il  itnons  i ons  is 
typio.illy  measured  by  ,i  high  oor ro  1 .it  i on  bot  wooti  v.iluos  on  .i 
psy o ho l og  l on  1 dimension  uni  v.iluos  on  t lio  onnd  iii.it  o ptiysto.il 
measure  , noross  st  imul  i . 

M OS  prooedures  ti.ivo  boon  used  suooosst'ully  to  identify 
psyoho log i on l dimensions  underlying  t tio  porooptton  of  spoooti 
so  Hints  (e.g.,  t'ols,  V . i n dor  k.imp,  .itnl  t’lomp,  19p9;  kloin, 
t'lomp,  Hint  t'ols,  1070;  Shepard,  197.';  Stuiw,  107*0;  oomplox, 
non- spoooti  sounds  l e.g.,  I’lomp  .uni  Htennoketi,  IOpO;  Mil  lor 
.itnl  fart orot t e , 10’s;  Moi>’,.in,  Woodho.nl,  .uni  Wobst  or , 107b; 

Howard  .uni  Silverman,  lo.’p;  tlow.nM,  .uni  oomplox 

vtsu.it  patterns  (e.g.,  Stonson,  19b8;  Shepard  .itnl  Chipmnn, 
1070;  Hard.’itiski  .itnl  1'nohel  l ,i , 10’’;  I'.iotioll.i  .itnl  Somers, 

19781.  In  ttiis  nontext  , suoooss  h.is  usu.illy  meant  ttmt  t tio 
derived  mu  1 t i 0 imons  i on.i  1 spaoo  .looount  s for  u largo 
proportion  of  tho  variability  in  ttio  similarity  Judgments 
atnl  ttiat  t tio  revealed  i.lont  ity  of  t tio  psy  o tio  log  i o.i  1 
.limotisiotis  is  intuit  ivolv  roasoimblo . 


Wo  suggest  tioro  that  mooting  oittior  or  bot  ti  of  tlioso 
or  it  or  la  .loos  not  provi.lo  strong  ovi.lonoo  to  support  ttio 
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similarity  judgments  in  one  task  can  then  be  used  to  predict 
behavior  in  some  other  , independent  task.  Our  present 
experiments  provide  an  example  of  one  such  test,  in  that  we 
use  the  MDS-derived  space  to  predict  performance  in  various 
identification  tasks. 

The  Identification  Model 

t- 

We  present  here  a decision  model  intended  to  predict 
the  confusion  matrix  for  a set  of  m stimuli  in  an 
identification  task,  on  the  basis  of  a multidimensional 
perceptual  space.  In  the  simplest  case,  the  perceptual 
space  is  that  revealed  by  application  of  an  MRS  procedure  to 
judgments  of  similarity,  and  we  shall  confine  our 
presentation  in  this  section  to  that  case.  We  note  for 
later  reference  that  the  decision  model  can  accept 
additional  dimensions,  as  suggested  by  other  evidence. 

We  take  as  our  starting  point  the  set  of  spatial 
coordinates,  v(> . . for  each  of  m stimuli,  s.  ( 1 £ i < m)  , on 
each  of  n dimensions,  d^  (1  < k < n) , as  provided  by  the 
MRS  procedure.  Though  the  model  can  be  used  with  any  MRS 
procedure  that  yields  such  a set  of  spatial  coordinates,  the 
data  analyses  presented  in  this  paper  are  based  on  the 
INRSCAl.  procedure  (Carroll  and  Chang,  1970;  Carroll,  1d7R; 
Carroll  and  Wish,  107^1.  INRSCAl  assumes  that  the  judged 
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similarity  between  any  pair  of  stimuli  is  a (decreasing) 
linear  function  of  the  Euclidean  inter st imul us  distance  in 
the  underlying  perceptual  space.  INDSCAL  differs  from  most 
other  MDS  procedures  in  that  it  yields  not  only  the  spatial 
conf iguration  of  the  set  of  m stimuli  in  the  n-d imensional 
"group  stimulus  space,"  but  also  a vector  of  weights  for 
each  observer  that  reflects  the  relative  importance  or 
salience  of  each  dimension  for  that  observer.  The  effect  of 
these  salience  weights  for  a given  observer  is  to  weight 
differentially  the  contribution  of  each  dimension  in 
determining  interst imulus  distance.  Specifically,  the 
distance  between  stimulus  and  stimulus  S^,  for  an 
observer  with  salience  weights  wk  ( 1 <.  k < n)  is  given  by 


D 


i , j 


V'i.k 


(1) 


Our  model  will  also  assume  the  weighted  Euclidean 
distance  metric  given  in  Equation  (1).  We  will  not  assume, 
however,  that  the  particular  set  of  salience  weights 
determined  for  each  observer  by  INDSCAL  in  the  similarity- 
judgment  task  necessarily  applies  to  the  identification 
tasks.  In  fact,  we  will  show  later  that,  within  observers, 
the  set  of  salience  weights  changes  in  predictable  ways 
across  different  conditions  of  the  identification  task. 
Accordingly,  the  salience  weights  for  each  observer  are 
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treated  in  the  model  as  parameters  whose  values  are  to  be 
estimated  from  the  confusion  data.  We  assume  further  that 
the  salience  weights  are  all  positive  (wk  >_  0)  and  sum  to 
' (£  wk  = The  ^atter  constraint  is  simply  a normalizing 
convention  which  reflects  the  fact  that  only  the  relative 
magnitudes  of  the  weights  are  meaningful  in  the  model. 


Having  defined  the  set  of  interstimulus  distances 

P.  , , the  next  step  is  to  relate  these  distances  to 
* • .1 

i nt er st imul us  confusab i 1 ity  in  the  identification  task.  On 
intuitive  grounds,  confusab i 1 i ty  should  be  some  monotone 
decreasing  function  of  interstimulus  distance.  We  define  a 
set  of  confusion  weights  C.  , , assuming  that  confusab i 1 i ty 

* t J 

between  stimulus  Sj  and  S,  is  given  by 


C . 

i ..1 


ex  p(  - aP  . ^ , 


(?) 


where  a 
decreases 
l n c r ease s 
decrease  s 


is  a sensitivity  parameter,  greater  than  0.  As  a 
towards  0,  over-all  stimulus  confusab i 1 i t y 
as  a becomes  large,  over-all  confusab i 1 i ty 


Several  consequences  of 
noting:  (11  with  both  a and  P. 
is  bounded  between  0 and  1;  (?) 


P 


i 


P ^ j , and 


this  relation  are  worth 


bounded  below  by  0,  C.  . 
t J l t J 

since  Pi  4 is  a distance 
■ Ci,.1  = C.i,i*  and 


t here  fore 
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since  = 0 for  all  i,  = 1 for  all  i.  The  choice  of 

this  particular  function,  from  several  considered,  was 
dictated  by  its  clear  superiority  in  accounting  for  our 


confusion  data  in  preliminary  analyses.  It  is  the  same 
assumption  used  successfully  by  Shepard  (1957,  1958a,  1958b) 
in  his  work  on  stimulus  and  response  generalization. 


i 


Finally,  the  conditional  probability  of  giving  the 
response  assigned  to  stimulus  S.  when  stimulus  was 
presented  is  assumed  to  be  the  confusabil ity  of  Sj  with  Sj 
relative  to  the  summed  con f usab i 1 i ty  of  all  stimuli  with  : 


i 

i 


Pr(Rj|S.) 


£ 

k 


C 


i,k 


(3) 


Equation  (3)  is  essentially  Luce's  choice  model  (1963),  with 
the  added  assumption  that  there  are  no  differential  response 
biases.  While  it  would  be  a simple  matter  to  include 
measures  of  response  bias  in  the  model , we  have  chosen  to 
exclude  them  here  for  reasons  of  simplicity  (fewer 
^parameters  to  estimate)  and  because  we  have  no  reason  to 
expect  strong  response  biases.  In  our  tasks,  the  a priori 
presentation  probabilities  (known  to  the  observers)  were 
equal  across  stimuli,  the  response  set  was  homogeneous,  and 
there  were  no  differential  payoffs. 
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I 

1 


We  are  also  implicitly  asserting  in  Equation  (3) 
that  the  set  of  responses  assigned  to  stimuli  are 
sufficiently  distinguishable  that  response  confusions  are 
negligible,  and  they  are  therefore  not  incorporated  into  the 
model . 

It  is  relatively  common  in  identification  tasks  to 
find  asymmetries  in  the  confusion  matrix  about  the  main 
diagonal;  that  is,  it  is  often  true  that 
Pr(R^|S.)  t Pr(R.|S.1.  One  well-known  source  of  this 
asymmetry  is  response  bias.  Of  interest  here  is  the 


observation  that  there  is  a second  possible  source  of 
confusion  asymmetries,  one  which  arises  from  the  decision 
rule  itself.  This  can  most  easily  be  seen  in  an  example, 
shown  in  Fig.  1,  in  which  three  stimuli  are  embedded  in  a 
two-dimensional  space.  As  drawn, 

D.  , > D.  0 > D0  , > P.  . = 0 (for  all  i), 
and  therefore 

C 1 , 3 < C 1 ,2  < C2,  ? < Ci,i  = 1 (for  aU  ' 
Calculating  Pr(R^|S?)  and  PriR^jS^I,  we  find 
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Figure  1.  Example  of  a 
asymmetric  co 
( see  tex  t)  . 
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and 


Pr(R?|S1) 


1 + 


Since  C,  , < C-,  ,,  we  may  conclude  that 

'id  £|  J 

Pr(R0|S^>  > Pr(R^|S2),  thus  demonstrating  asymmetry  in  the 
confusion  matrix  about  the  main  diagonal. 

From  this  example,  we  can  see  that  the  confusion 
asymmetry  between  two  stimuli  S^  and  S^  arises  because  the 
two  response  probabilities  Pr(R.|S.l  and  Pr(R.|S.)  are 
determined  by  the  magnitude  of  the  shared  confusion  weight, 
relative  to  a sum  of  confusion  weights,  where  the  sum 
is  determined  from  the  "point  of  view"  of  the  stimulus,  S- 
or  S^.  These  two  sums  may  differ  substant i al 1 y in 
magnitude,  depending  upon  the  geometric  configuration  of  the 
stimuli  in  the  perceptual  space. 


Two  Types  of  Identification  Task 

The  primary  test  of  the  ability  of  the  MBS-derived 
perceptual  space,  and  the  associated  decision  model,  to 
predict  identification  behavior  involves  a straightforward 
ident i f icat ion  task  in  which  m responses  are  paired 
(one-to-one!  with  the  m stimuli  presented.  We  call  this  a 
"complete"  identification  task. 


\ 


12 


Report  No . 3711 


Bolt  Reranek  and  Newman  Inc. 


We  also  consider  a "partial"  identification  task  in 
which  fewer  than  m responses  are  available  for  use  with  the 
m stimuli  presented.  This  task  corresponds  to  a situation 
in  which  some  subset  of  the  total  number  of  stimuli  present 
is  of  special  interest  --  the  members  of  this  subset 
constituting  "signals"  --  while  the  remaining  stimuli  are 
regarded  as  "noise,"  and  as  not  requiring  identification. 
We  have  referred  to  this  class  of  task  elsewhere  as  a 
"detect i on-and- id ent i f icat ion"  task  (Swets,  Green,  Getty, 
and  Swets,  11771.  The  observer  is  asked  first  to  make  a 
detection  response  ("signal"  or  "noise")  and  then  to  choose 
one  of  the  available  identification  responses  --  the  one 
corresponding  to  the  one  of  the  signals  most  likely  to  be 
present  . 

The  partial  identification  task  employed  here  had 
three  conditions,  with  a different  subset  of  the  m stimuli 
defined  as  signals  in  each  condition.  This  variation  across 
conditions  reflects  the  practical  fact  that  the  subset  of 
stimuli  that  is  of  special  interest  varies  from  one 
situation  to  another.  The  listener  to  degraded  speech  may 
want  to  concentrate  at  some  time  on  distinguishing  between 
.lust  two  particular  phonemes,  rather  than  among  thirty  or 
so.  The  sonar  observer  may  wish  to  distinguish  among  ships 
and  ignore  variations  in  ocean  depth,  or  vice  versa.  In  t iie 

1 3 


f 


ft 
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present  instance,  t lie  variation  across  conditions  provides  a 
test  of  our  decision  model's  ability  to  deal  with  changes  in 
the  salience  weights,  which  may  reflect  changes  in  the 
relative  usefulness  of  the  various  perceptual  dimensions  as 
the  set  of  signals  is  changed. 

Met  hod 


St  i m u H 

Our  stimuli  consisted  of  visual  representations  of  a 
set  of  eight  underwater  sounds,  originally  selected  by 
Howard  (1977)  to  represent  a range  of  eonfusible  natural  and 
meohan ica 1 1 y- produced  sounds.  They  were  referred  to  by 
Howard  as  ( 1 1 Sheet  Cavitation  (SC!,  (01  Biologies  (HP,  (P 
Compressed  Cavitation  (CC),  ( H)  Torpedo  (TO),  (R!  Piesel 
Fngine  (PF!,  (t ' Rain  Squall  (RS),  (7'  Steam  Noise  (SN',  and 
(SI  Flutter  (FI.!.  Their  long-term  energy  spectra  are  shown 
in  Fig.  0 . 


Our  visual  representations  displayed  the  spectra  as 
frequency  (horizontal  axis!  versus  time  (vertical  axis! 
versus  energy  (darkness  --  the  greater  the  energy,  the 
darker  the  trace!.  We  introduced  periodicity  as  an 
additional  physical  dimension  by  sinusoidally  varying  the 
average  darkness  of  the  signal  profile  in  the  temporal 
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Figure  2.  Long-term  spectra  of  eight  underwater  sounds 
(From  Howard,  19761. 
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(vertical)  direction.  We  grouped  (1)  SC  and  (2)  BI  with  (7) 
SN  and  (ft)  FL,  giving  all  four  relatively  low-frequency 
periodicity,  and  gave  the  remaining  four  relatively 
high-frequency  periodicity.  Specifically,  the  cycles  per 
stimulus  were  IS,  1 6 , 17,  and  18  for  Stimuli  1,  2,  7,  and  8, 
respectively,  and  21,  22,  23,  and  24  for  stimuli  3,  4,  5, 
and  6,  respectively.  The  resulting  visual  patterns  are 
shown  in  Fig.  ft. 


The  stimuli  were  constructed  on  a COMTAL  model 
8000-SA  image-processing  system,  driven  by  a DEC  PDP-11/34 
minicomputer,  and  displayed  in  an  area  24  cm  wide  by  12  cm 
high  on  a CONRAC  17-inch  (43-cm)  SNA  television  monitor.  As 
is  apparent  in  Fig.  3,  we  added  a background  of  random  noise 
to  each  stimulus  pattern.  The  noise  consisted  of  a 
256  x 128  matrix  of  elements,  each  having  an  independent 
gray  value  sampled  from  a Gaussian  distribution  with  mean 
128  units  and  standard  deviation  15  units  on  the  256  unit 
gray  scale  of  the  COMTAL. 


Each  stimulus  pattern  was  constructed  by  subtracting 
from  the  noise  background,  sampled  anew  on  each  trial,  the 
horizontal  brightness  profile  corresponding  to  its  long-term 
spectrum.  Thus,  increasing  energy  in  the  spectrum  resulted 
in  a darker  trace.  The  spectral  profiles  of  all  eight 
signals  were  scaled  to  have  the  same  space-average  darkness 
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Figure  3.  Visual  representations  of  the  eight  underwater 
sounds,  photographed  from  the  display  monitor. 
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of  20  gray  units  below  the  mean  gray  value  of  the  noise 
background.  In  addition,  the  darkness  of  each  point  in  a 
signal's  profile  was  sinusoidally  varied  in  the  vertical 
dimension.  For  the  s im il ar i ty- judgment  task,  the 
peak-to-peak  brightness  variation  was  200  percent  about  the 
steady-state  value,  resulting  in  h igh- contrast  images 
similar  to  those  shown  in  Fig.  3.  For  the  various 
identification  tasks,  the  peak-to-peak  brightness  variation 
was  reduced  to  60  percent  about  the  steady  state  value, 
thereby  reducing  the  image  contrast  cons iderabl y . 

For  all  tasks,  the  brightness  and  contrast  controls 

on  the  CONRAC  monitor  were  adjusted  such  that  the  middle 

2 

gray  (128  units)  had  a luminance  of  about  62  cd/m  , and  full 

2 

white  (255  units)  a luminance  of  about  308  cd/m  . 

Apparatus 

In  the  various  identification  tasks,  three  observers 
sat  at  individual  video  computer  terminals  (Lear  Siegler 
ADM-3A)  approximately  two  meters  from  the  stimulus-display 
screen,  whose  center  was  about  1.1  m above  the  floor. 
Ambient  room  lighting  was  maintained  at  a dim  level. 


All  experimental  events  (stimulus  display,  response 
recording,  and  trial  timing)  were  controlled  by  the  PDP-11 
computer . 
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Proced  ure 

Similarity  Task  . The  observers  rated  similarit  y of 
pairs  of  stimuli  on  a 10-point  scale.  Each  of  the  08 
possible  pairs  was  presented  side  by  side  by  means  of  35mm 
slides  for  15  seconds,  followed  by  a 15-second  response 
interval.  Each  slide  was  a photograph  of  an  high  contrast 
display  of  one  of  the  eight  stimuli  on  the  CONRAC  monitor 
screen,  similar  to  Fig.  3.  Other  details  of  the  procedure 
are  available  elsewhere  (Swets,  Green,  Getty,  and  Swets, 
15771  . 

Compt  ete Identification  Task.  Each  trial  began  by 

blanking  of  the  COMTAL  screen,  followed  2.5  seconds  later  by 
a 2.0  second  low-contrast  display  of  one  of  the  eight  visual 
stimuli.  Each  observer  then  made  a self-paced 
identification  response  on  the  terminal  keyboard,  pressing 
one  of  eight  keys  labelled  with  the  digits  1 to  8 (typing 
errors  could  be  corrected  with  an  "erase"  key).  Observers 
could  make  reference  to  a sheet  on  which  were  arranged 
labelled,  high-contrast.  Polaroid  photographs  of  the  eight 
stimuli  --  again,  similar  to  those  shown  in  Fig.  3.  As  each 
observer  responded,  the  number  of  the  presented  signal  was 
displayed  on  his/her  terminal's  screen.  When  all  three  had 
responded,  the  stimulus  image  was  redisplayed  along  with  the 
correct  answer  for  about  two  seconds. 
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The  sequence  of  stimuli  was  determined  completely  at 
random.  Fifty  trials  were  presented  in  a block,  and  three 
blocks  were  presented  in  a one-hour  session. 

Partial  Identification  Task.  In  this  task,  four  of 
the  eight  stimuli  were  designated  as  "signal"  and  allowed  as 
responses,  while  the  remaining  four  were  designated  as 
"noise"  and  not  allowed  as  responses.  A different  set  of 
four  stimuli  was  designated  as  the  "signal"  set  in  each  of 
three  conditions.  The  procedure  for  the  partial 
identification  task  was  generally  the  same  as  that  described 
above  for  the  complete  identification  task  except  that,  for 
reasons  discussed  elsewhere  (Swets,  Green,  Getty,  and  Swets, 
1977),  on  each  trial  a stimulus  pattern  was  presented  in 
five  sequential  stages.  At  each  stage,  a successive  fifth 
of  the  pattern  was  revealed,  pushing  down  the  display  of 
earlier  stages,  and  the  observer  made  both  a detection  and 
an  identification  response.  Our  present  analyses  use  only 
the  identification  responses  from  the  last  (fifth)  stage  of 
each  trial  when  the  complete  display  of  the  stimulus  was  in 
view.  Observers  were  provided  feedback  --  either  the  signal 
number  (when  one  of  the  four  "signals"  was  presented)  or  the 
word  "noise"  (when  one  of  the  four  "noise"  stimuli  was 
presented)  --  at  the  end  of  each  trial. 
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Observers 

The  three  observers  were  members  of  BBN's  technical 
staff,  including  one  of  the  experimenters  (JBS). 

Derivation  of  the  Perceptual  Space: 

TTTe  Similarity-Judgment  TasF 

A set  of  28  similarity  judgments  (all  pairs  of  the 
eight  stimuli)  for  each  of  the  three  observers  was  submitted 
to  INDSCAL  analysis.  The  psychological  coordinates  of  the 
eight  stimuli  in  the  three-dimensional  perceptual  space 
revealed  by  INDSCAL  are  listed  as  tp  ■ . to  ip . in  Table  1. 

Details  of  the  INDSCAL  analysis  are  available  elsewhere 
(Swets,  Green,  Getty,  and  Swets , 1977).  Briefly,  we  found 
the  three  INDSCAL-der ived  dimensions  to  correlate  highly 
with  physical  measures  of  "low-frequency  energy," 
"m id- frequency  energy,"  and  "contrast,"  respectively. 

Surprisingly,  "periodicity"  did  not  emerge  as  a 
psychological  dimension,  despite  the  fact  that  our  observers 
reported  it  to  be  a salient  dimension  in  the  identification 
task.  Moreover,  as  described  in  the  earlier  report,  an 
INDSCAL  analysis  applied  to  the  similarity  judgments  of 
another  group  of  14  judges  yielded  periodicity  as  a fourth 
dimension,  along  with  the  same  three  dimensions  as  given  by 
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In  order  to  determine  the  gain,  if  any,  in 
predicting  confusion  matrices  when  periodicity  was  included 
as  a fourth  dimension  for  the  three  observers,  we  created  a 
periodicity  coordinate  for  each  stimulus.  The  assigned 
values,  given  in  the  last  column  of  Table  1,  are  linearly 
related  to  our  physical  measure  of  periodicity,  subject  to 
the  constraints  --  used  by  INDSCAL  in  assigning  coordinates 
on  a psychological  dimension  --  that  (1)  the  mean  across 
stimuli  is  0,  and  (2)  the  variance  across  stimuli  is  1.  We 
realize  that  the  true  psychological  coordinates  for 
periodicity  are  probably  not  linear  with  the  physical 
measure;  however,  lacking  a strong  rationale  for  any  other 
specific  relationship,  we  may  suppose  that  linearity  is  a 
good  first  approximation  to  the  true  relationship. 

The  8x8  Complete  Identification  Task 

There  were  eight  response  alternatives  in  the 
complete  identification  task,  each  corresponding  to 
identification  of  one  of  the  eight  stimulus  patterns.  Of 
the  51*  blocks  of  trials  run,  the  first  three  were  regarded 
as  practice  and  omitted  from  analysis.  In  addition,  the 
data  of  two  other  blocks,  and  part  of  a third,  were  lost  due 
to  equipment  failures.  The  remaining  2421  trials  for  each 
observer  were  included  in  the  analyses  that  follow. 
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Re  suits 


Error  Probability.  A plot  of  error  probability 

against  stimulus  number,  shown  for  individual  observers  in 
Fig.  *1 , reveals  that  the  stimuli  were  not  equally 
confusible.  In  fact.,  Stimulus  No.  2 (PI)  was  never,  or 
almost  never,  confused  with  any  other  stimulus.  Individual 
observers  showed  similar  patterns  of  errors  across  stimuli, 
as  seen  in  Fig.  . They  also  showed  similar  over-all  error 
rates  (1?,  20,  and  21  percent  errors  for  observers  BF,  JK, 

and  JS,  respectively).  The  over-all  probability  of  a 

confusion  error,  averaged  across  stimuli  and  observers,  was 
18  percent  ( 1 1fl'l  errors  in  7263  trials). 

Confusion  Matrix.  The  matrices  of  raw  confusion 

frequencies  are  given  for  each  observer  in  Table  2.  The 

patterns  of  confusions  embedded  in  these  numbers  are  most 
readily  apparent  when  response  probability  distributions  are 
plotted  for  each  stimulus,  shown  separately  for  each 
observer  by  the  solid  lines  and  filled  circles  in  Fig.  5.  A 
prominent  feature  of  these  data  is  the  high  degree  of 
similarity  across  the  three  observers  in  the  confusion 

pattern  for  each  stimulus.  It  is  this  matrix  of  confusion 

distributions  that  we  seek  to  predict  by  the  model,  as 

described  in  the  following  section. 
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Table  0 

8x8  Experiment:  Matrix  of  Confusion 
Frequencies  for  Each  Observer 
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8x8  EXPERIMENT 
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Figure  BA.  3x8  Experiment:  Distribution  of  response 

probability  for  each  stimulus  (Stimuli  1 to  , for 
each  observer.  Obtained  distributions  are  given  by 
solid  lines  and  filled  circles;  distributions  predicted 
by  the  model  are  given  by  dashed  lines  and  open  circles. 
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8x8  EXPERIMENT 


0 BF  0 JK  0 JS 


RESPONSE  NUMBER 

Figure  5B  . 8 x 9 Experiment:  Distribution  of  response 

probability  for  each  stimulus  (Stimuli  5 to  81,  for 
each  observer.  Obtained  distributions  are  given  by 
solid  lines  and  filled  circles;  distributions  predicted 
by  the  model  are  given  by  dashed  lines  and  open  circles. 
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Model  Analysis 

Parameter  Estimation.  Model  parameters  were 
estimated  separately  for  each  observer  using  a 
negative-gradient  h i 1 1 -cl imb ing  technique  that  sought  to 
minimize  the  sum  of  squared  deviations  between  the  predicted 
and  obtained  confusion  matrices  for  that  observer.  Under 
several  different  analysis  conditions  regarding  the 
dimension  salience  weights,  discussed  below,  the  number  of 
parameters  estimated  ranged  from  one  (a,  the  sensitivity 
parameter)  to  five  (a  and  four  salience  weights,  w1  to  wy). 

Prediction  of  Confusion  Matrices.  We  first  fitted 
the  model  to  the  individual  confusion  matrices  of  our  three 
observers  using  the  three  dimension  salience  weights  for 
each  observer  provided  by  the  INDSCAL  analysis.  The 
parameter  values  of  individual  observers,  and  the  proportion 
of  variance  accounted  for  in  the  data  of  individual 
observers,  and  the  average  values,  are  shown  in  Table  7A  . 

Estimating  only  a single  parameter,  a,  for  each 
observer,  we  found  that  we  were  able  to  account  for  97 
percent  of  the  variance  in  the  full  confusion  matrix, 
averaged  across  observers.  On  the  other  hand,  if  we 
included  only  identification  errors  (the  off-diagonal 
elements)  in  the  variance  analysis,  we  accounted  for  only  21 
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Table 


8 x 8 Experiment:  Estimated  Parameter  Values 
and  the  Proportion  op  Variance  Accounted  for  by 
the  Model,  for  Each  Observer 


Proportion  of  Vai 
Accounted  Foi 
Full  Matrix  I Error 


A.  Three-dimensional  model  : w,-w  INDSCAL  constrained 


■ . Three-dimensional  model 


Four-dimens lonal  model 


INDSCAL  constrained 


our-dimens ional  model  : w,-w,  free  parameters 


parameter  not  estimated;  value  derived  from  INDSCAL  analysis 


.54 

.40 

.43 

9.4 

7.5 

7.3 

.46 

8.1 

• 73 
.55 
.49 

. 14 

.27 

.28 

• 3 

. 18 

• 23 

.59 

• 23 

.18 
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percent  of  the  error-matrix  variance.  There  are  at  least 
four  reasons  why  the  variance  accounted  for,  considering 
only  errors,  may  be  small;  we  take  them  up  in  turn  in  the 
next  four  paragraphs. 

First,  the  obtained  relative  frequencies  calculated 
for  the  off-diagonal  cells  contain  the  effects  of  both 
quantization  error  (the  observed  frequency  of  each  response 
is  integral)  and  sampling  error.  Since  the  predicted 
probabilities  of  most  off-diagonal  cells  are  very  small,  the 
range  of  variation  in  the  measures  to  be  correlated  is 
small.  We  may  expect,  on  these  grounds,  that  the 
unpredicted  variation  due  to  quantization  and  sampling  error 
is  substantial  relative  to  the  small  total  amount  of 
variation  present. 

Second,  the  brief  stimulus  duration  (2  seconds)  may 
have  resulted  in  some  proportion  of  trials  on  which  the 
stimulus  either  was  not  seen,  or  not  seen  long  enough  for 
adequate  encoding.  Responses  on  these  trials  should 
represent  pure  guesses,  uniformly  distributed  across  the  set 
of  responses  (assuming  no  response  biases).  While  guessing 
undoubtedly  occurred  occasionally,  it  is  unlikely  that  it 
represents  a major  contribution  to  the  confusion  matrix.  If 
it  did,  we  would  expect  to  see  a non-zero  baseline  response 
frequency  across  all  responses  for  a given  stimulus,  an 
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expectation  which  is  not  confirmed  by  the  observed  confusion 
matrices  . 


> 


Third,  the  set  of  dimension  salience  weights  derived 
by  INDSCAL  from  the  s im i 1 ar i t y- j udgmen t task  may  not 
accurately  represent  dimension  salience  in  the 
identif ication  task.  This  possibility  was  examined  by 
fitting  the  model  with  the  salience  weights,  w^  to  w^,  and 
the  sensitivity  parameter,  a,  free  to  vary.  As  seen  in 
Table  3B,  the  estimated  parameter  values  change  somewhat 
from  their  I NDSCAL-der ived  values;  however,  there  is  no 
significant  increase  in  proportions  of  variance  accounted 
for  either  in  the  full  matrix  or  in  the  off-diagonal  cells. 
Thus,  for  a model  of  three  dimensions,  the  I NDSCA L-der i v ed 
salience  weights  are  nearly  optimal  relative  to  the  best 
possible  performance  of  the  model  with  unconstrained  choice 
of  parameter  values. 


This  leads  us  to  consider  the  fourth  possible 
reason,  namely,  that  one  or  more  dimensions  used  by  the 
observers  in  the  identification  task  did  not  emerge  in  the 
INDSCAL  analysis.  As  discussed  earlier,  we  have  good  reason 


to  suspect 

that  the 

temporal  periodicity 

present 

in  the 

st imulus 

patterns 

i s 

one  such 

missing 

dimension. 

So  we 

fitted  the 

model  to 

the 

conf  usion 

matrices 

a third 

time. 

using  the  INDSCAL-der ived  salience  weights  for  each  observer 
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for  the  first  three  dimensions,  and  allowing  the  salience 
weight  for  the  fourth  dimension  (periodicity)  to  vary.  By 
including  periodicity  as  a fourth  dimension,  we  found  that 
the  average  proportion  of  variance  accounted  for  in  the  full 
matrix  increased  from  97  percent  to  98  percent , shown  in 
Table  3C , and  that  accounted  for  in  the  off-diagonal  cells 
increased  from  21  percent  to  39  percent,  a substantial 
improvement.  Moreover,  with  an  average  salience  weight  of 
.46  assigned  to  periodicity,  it  was  by  far  the  most 
important  dimension  among  the  four  in  determining 
i nter st imul us  distance. 

Finally,  we  wished  to  determine  if  any  further 
improvement  in  prediction  might  be  obtained  by  allowing  the 
salience  weights  (constrained  to  their  INPSCAL  values  in  the 
last  fit)  to  vary.  The  result,  shown  in  Table  3D,  was 
essentially  no  further  increase  in  the  proportion  of 


variance  accounted  for. 

The  values  of 

w 1 * 

w2’ 

and  w , 

changed  very  little, 

on  average, 

f r om 

the 

INPSCAL 

values  --  indicating,  as 

observed  before, 

that 

the 

1 NPSCAl 

values  were  nearly  optimal  in  terms  of  the  model's  ability 
to  predict  the  full  confusion  matrix. 

The  pattern  of  results  discussed  above  suggests  that 
the  effect  of  including  periodicity  as  a fourth  dimension  is 
independent  of,  and  additive  to,  the  effect  of  freely 
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estimating  the  first  three  salience  weights.  Including 
periodicity  as  a fourth  dimension  increases  the  proportion 
of  variance  accounted  for  by  1 percent  and  18  percent, 
respectively,  in  the  full  and  error  matrices  when  INDSCAL 
constraints  are  used  (Tables  3A  and  3 C),  and  by  1 percent 
and  22  percent  in  the  full  and  error  matrices  when 
parameters  are  freely  estimated  (Tables  3B  and  3D).  Freely 
estimating  weights  w^  to  w^  increases  the  proportion  of 
variance  accounted  for  by  0 percent  in  both  the  full  and 
error  matrices  when  only  three  dimensions  are  used  (Tables 
3A  and  3R),  and  by  0 and  9 percents  when  periodicity  is 
included  as  a fourth  dimension  (Tables  3C  and  3D). 

We  turn  now  from  summary  measures  of  goodnes  s-o  f-f  i t. 
to  the  prediction  of  individual  cells  in  the  confusion 
matrix,  using  the  four-dimensional  model  with 
freely-estimated  salience  parameters.  We  choose  this 
version  of  the  model,  in  spite  of  the  almost  equally  good 
fit  noted  above  when  INDSCAL  constrained  parameters  are 
used,  for  consistency  with  data  reported  in  the  next  section 
on  the  partial  identification  tasks,  where  fits  of  the 
different  model  versions  were  not  equally  good.  The 
distributions  of  predicted  conditional  response 
probabilities  are  plotted  for  each  observer  in  Fig.  B as 
open  circles  connected  by  dashed  lines,  superimposed  on  the 
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obtained  distributions.  The  difficulty  one  has  in 
separating  obtained  and  predicted  curves  attests  to  the 
considerable  accuracy  of  the  predictions. 

There  are  occasional  deviations  between  predicted 
and  obtained  probabilities  that  appear  to  be  systematic,  in 
that  two  of  the  three  observers  show  the  same  pattern  of 
deviations.  For  example  when  Stimulus  3 was  presented,  both 
observers  JK  and  JS  made  Response  6 more  frequently  and 
Response  7 less  frequently  than  predicted.  This,  and  other 
such  examples,  may  result  from  the  observers'  use  of 
dimensions  that  were  available  in  the  set  of  patterns  but 
not  included  in  the  model  analyses.  Over-all,  though,  the 
model  that  incorporates  three  I NPSCAL-der i ved  dimensions, 
and  a fourth  added,  predicts  quite  well  the  individual  cells 
of  the  confusion  matrix. 


The  8 x 4 Partial  Identification  Task 

We  may  test  the  model  in  another  way  by  considering 
how  well  it  is  able  to  predict  the  pattern  of  identification 
confusions  when  an  observer  is  limited  to  responses 
associated  with  only  a subset  of  the  eight  stimuli.  In  this 
section  we  apply  the  model  to  three  conditions  of  an  8 x 4 
partial  identification  task,  in  which  only  four  of  eight 
stimuli  --  referred  to  as  the  "signals"  --  correspond  to 
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allowable  ident 1 f icat Ion  responses,  a different  set  of  four 
stimuli  in  each  of  the  three  conditions. 

The  "signals"  in  Condition  1 were  the  four  stimuli 
1,  2,  5,  and  6 of  Fig.  3.  These  signals  were  not  clearly 
distinguished  from  the  remaining  four  "noise"  stimuli  on  any 
of  the  four  physical  dimensions  discussed  previously.  In 
Condition  2,  the  signals  were  stimuli  3,  5,  and  6,  the 
patterns  with  relatively  high-frequency  periodicities.  In 
Condition  3,  the  signals  were  stimuli  1,  3,  5,  and  7, 
patterns  which  tended  to  have  low  values  on  our  physical 
measure  of  m id- frequency  energy. 

The  analyses  that  follow  are  based  on  ?30  trials  for 
each  observer  in  each  condition.  On  the  average,  each  of 
the  eight  stimuli  was  presented  about  08  times  in  each 
condition.  An  initial  block  of  10  practice  trials  has  been 
omitted  from  analysis  for  each  condition. 

Rosu  Its 

Error  Probability.  Error  rates  can  be  defined  only 
for  the  four  "signals"  in  each  condition.  All  responses  on 
"noise"  trials  were  necessarily  errors  since  the  noise 
stimuli  did  not  correspond  to  allowable  identification 
responses.  Using  only  the  signal  subsets  of  the  confusion 
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matrix  for  each  condition,  we  found  considerable  variation 
in  the  error  rates  of  the  three  observers,  averaged  across 
conditions:  8 , 24,  and  8 percent  errors  for  observers  BF, 
JK,  and  JS,  respectively.  There  was  also  variation  in  the 
error  rates  of  the  three  conditions,  averaged  across 
observers:  8,  21,  and  9 percent  for  Conditions  1,  2,  and  3, 
respectively.  Finally,  the  over-all  error  rate  in  this 
experiment,  17  percent,  was  somewhat  lower  than  that  in  the 
full  8x8  experiment  (18  percent). 


Confusion  Matrices.  The  raw  confusion  matrices  are 
given  in  Table  4 for  each  observer  and  for  each  condition. 
The  corresponding  response  distributions  are  plotted  in 
Figs.  6-8  for  each  stimulus,  for  each  observer,  and  for  each 
condition,  by  the  filled  circles  connected  by  solid  lines. 
As  in  the  first  experiment,  the  response  distributions  for  a 
given  stimulus  and  condition  are  generally  very  similar 
across  the  three  observers.  The  most  notable  exceptions  are 
the  distributions  for  stimuli  6,  7,  and  8 in  Condition  2 
(Fig.  7B1 , and  for  stimuli  2,  6,  and  8 in  Condition  3 (Figs. 
8A  and  8B> . With  the  exception  of  stimulus  6 in  Condition 
2,  these  are  all  instances  in  which  the  identification 
response  corresponding  to  the  presented  stimulus  was  not 
among  the  set  of  allowed  responses,  a fact  we  will  return  to 
shortly  in  considering  the  model's  predictions. 
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8x4  CONDITION  1 


OBF  O’JK  OJS 


1256  1256  1256 


RESPONSE  NUMBER 

Figure  6A.  8 x U Experiment,  Condition  1:  Distribution  of 

response  probability  for  each  stimulus  (Stimuli  I to  ^ 
for  each  observer.  Obtained  distributions  are  given  by 
solid  lines  and  filled  circles;  predicted  distributions 
are  given  by  dashed  lines  and  open  circles. 
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8x4  CONDITION  2 


3 4 5 6 


3 4 5 6 

RESPONSE  NUMBER 


3 4 5 6 


3x4  Experiment,  Condition  2:  Distribution  of 
response  probability  for  each  stimulus  (Stimuli  1 to  4 ) 
for  each  observer.  Obtained  distributions  are  given  by 
solid  lines  and  filled  circles;  predicted  d i str ibut  ions 
are  given  by  dashed  lines  and  open  circles. 


conditional  response  probability 
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8x4  CONDITION  3 

OBE  OJK  OJS 


1357  1357  1357 

RESPONSE  NUMBER 


Figure  8fl.  8 x M Experiment,  Condition  3:  Distribution  of 

response  probability  for  each  stimulus  (Stimuli  1 to  Ml, 
for  each  observer.  Obtained  distributions  are  given  by 
solid  lines  and  filled  circles;  predicted  distributions 
are  given  by  dashed  lines  and  open  circles. 


CONDITIONAL  RESPONSE  PROBABILITY 


Report  No.  3719  Bolt  Beranek  and  Newman  Inc. 

8x4  CONDITION  3 

OBF  0 JK  O JS 


1357  1357  1357 


RESPONSE  NUMBER 

Figure  8B.  8x9  Experiment,  Condition  3:  Distribution  of 

response  probability  for  each  stimulus  (Stimuli  5 to  8) , 
for  each  observer.  Obtained  distributions  are  given  by 
solid  lines  and  filled  circles;  predicted  distributions 
are  given  by  dashed  lines  and  open  circles. 
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Model Anal  ys  is 


Parameter  Estimation.  The  application  of  the  model 
to  the  confusion  matrices  proceeded  in  much  the  same  way  as 
in  the  previous  experiment.  As  before,  the  model  was  fitted 
to  the  data  under  four  different  conditions  corresponding  to 
combinations  of  two  factors:  (1)  the  inclusion  or  not  of 
periodicity  as  a fourth  dimension,  and  (?)  the  use  of 
1 NDSCAL-der i ved  or  freely-estimated  values  for  the  three 
salience  weights,  w^  to  w..  In  all  cases,  the  psychological 
coordinates  of  the  eight  stimuli  on  the  first  three 
dimensions  were  taken  from  our  INDSCAL  analysis  of  the 
s imi larity- judgment  data,  and  the  coordinates  of  periodicity 
from  our  physical  measure  of  that  dimension.  Model 
parameters  were  estimated  separately  for  each  observer  in 
each  of  the  three  experimental  conditions,  and  for  each  of 
the  four  versions  of  the  model. 


Predictions  of  Confusion  Matrices 


The  parameter 


values  and  pro  port i on-o f-var i ance  values  for  each  condition, 
averaged  across  observers,  and  the  average  values  across 
conditions,  are  shown  for  each  of  the  four  versions  of  the 
model  in  Table  b . Comparing  the  average  proportion  of 
variance  accounted  for  by  each  model  in  Table  5 with  the 
comparable  average  value  for  the  8 x 8 experiment  in  Table 
1,  it  is  clear  that  the  model  is  accounting  for  somewhat 
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less  of  the  variance  in  the  full  matrix  than  before,  but  for 
substantially  more  of  the  variance  in  the  error  matrix.  The 
first  result  is  probably  due  in  large  part  to  the 
considerably  smaller  number  of  trials  contributing  to  each 
of  the  confusion  matrices  --  approximately  30  --  resulting 
in  less  stable  estimates  of  the  observed  conditional 
response  probabilities.  The  second  result  follows  from  the 
inclusion  of  noise  trials  on  which  observers  are  forced  to 
make  confusion  errors.  As  a consequence,  the  amount  of 
variability  available  to  be  explained  is  large  in  the  error 
matrices  of  the  8 x '1  experiment  relative  to  the  amount  in 
the  error  matrices  of  the  8x8  experiment. 

Paralleling  the  results  of  the  analysis  of  the  8x8 
experiment,,  the  pattern  of  changes  in  the  variance  accounted 
for  across  the  four  versions  of  the  model  suggests  that  the 
effect  of  including  periodicity  as  a fourth  dimension  is 
independent  of,  and  additive  to,  the  effect  of  freely 
estimating  the  first  three  salience  weights.  Including 
periodicity  as  a fourth  dimension  increases  the  proportion 
of  variance  accounted  for  by  B and  10  percent  in  the  full 
and  error  matrices,  respectively  (Tables  BA  and  BC ) . Freely 
estimating  the  salience  weights  w(  to  w>  increases  the 
proportion  of  variance  accounted  for  by  10  and  ?0  percent  in 
the  full  and  error  matrices,  respectively  (Tables  BA  and 
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5 B ) . Doing  both  --  including  periodicity  as  a fourth 
dimension  and  freely  estimating  the  salience  weights  w1  to 
w,  --  increases  the  proportion  of  variance  accounted  for  by 
16  and  30  percent  in  the  full  and  error  matrices, 
respectively  (Tables  5A  and  5P). 

The  substantial  improvement  in  the  model's 
predictions,  when  the  salience  weights  w1  to  w , are  freed  of 
their  INDSCAL-der ived  values  and  independently  estimated  in 
each  condition,  is  in  contrast  to  the  lack  of  improvement 
observed  in  the  8x8  experiment.  The  reason  is  apparent  in 
Fig.  9,  which  shows  the  estimated  salience  weights 
(including  the  periodicity  dimension)  for  each  observer, 
separately  for  the  8x8  experiment  and  each  of  the  three 
conditions  of  the  8x4  experiment.  The  pattern  of 
estimated  weights  clearly  changes  from  one  condition  to 
another,  suggesting  that  the  observers  modified  their  set  of 
salience  weights  from  condition  to  condition  according  to 
the  composition  of  the  set  of  four  stimuli  defined  as 
signals.  In  Conditions  1 and  2,  the  patterns  of  weights 
appear  quite  similar  across  observers;  in  Condition  3,  they 
appear  relatively  different. 

The  predicted  response  distributions  for  each 
stimulus  and  for  each  observer  in  each  of  the  conditions  are 
shown  by  the  open  circles  connected  by  dashed  lines  in  Figs. 
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6-8.  These  predictions  are  based  on  the  four-dimensional 
model  with  freely  estimated  salience  weights.  Scanning  over 
the  large  number  of  distributions,  the  over-all  impression 
is  that  there  is  a remarkably  good  agreement  between 
predicted  and  obtained  distributions.  It  is  worth  noting 
that  a ten-percent  deviation  between  obtained  and  predicted 
probabilities  corresponds  to  a difference  of  only  about 
three  responses,  given  that  each  obtained  distribution  is 
based  on  about  30  trials. 

We  mentioned  earlier  that  most  of  the  cases,  in 
which  the  observed  response  distributions  for  a given 
stimulus  differed  among  the  three  observers,  were  instances 
of  noise  stimuli.  We  now  observe  that  most  of  the  cases  in 
which  there  is  a substantial  deviation  between  predicted  and 
obtained  response  distributions  are  also  instances  of  noise 
stimuli  (e.g.,  Stimulus  6 for  Observer  BF  in  Condition  31- 
In  fact,  the  structure  of  the  model  provides  an  insight  into 
the  source  of  these  deviations.  It  will  sometimes  be  true 
that  the  interstimulus  distances  between  a particular  noise 
stimulus  and  each  of  the  four  signal  stimuli  allowed  as 
responses  are  all  very  large,  and  thus  that  the 
corresponding  con fusab i 1 it y weights  are  all  very  small.  The 
observer  would  have  no  difficulty  in  rejecting  all  four- 
responses  as  correspond  ing  to  the  true  "identity"  of  the 
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presented  stimulus.  In  this  predicament  the  observer  most 
likely  reinterprets  the  task  to  respond  with  the  stimulus 
among  the  allowable  four  which  is  "most  similar"  to  the 
presented  stimulus,  even  though  it  is  clearly  incorrect. 
However,  if  the  confusabil ity  weights  for  all  four  responses 
are  extremely  small,  we  may  expect  the  observed  response 
distribution  to  be  very  sensitive  to  small  differences  in 
salience  weights,  and  also  to  other  decision  processes. 

A particularly  good  illustration  of  the  problem  is 
found  in  the  response  distributions  to  noise  Stimulus  2 in 
Condition  3 (Fig.  8).  Each  observer  made  a single  response 
exclusively  --  but  a different  response  for  each  observer. 
In  all  likelihood,  each  observer  was  aware  that  the  pattern 
was  Stimulus  2 at  each  presentation.  Having  decided  once 
which  signal  stimulus  was  "most  similar"  to  Stimulus  2,  the 
observer  then  remembered  and  gave  that  response  consistently 
thereafter.  It  can  be  seen  in  Fig.  8 that  the  model 
accurately  predicted  the  different  distributions  for  the 
three  observers  --  but  at  a cost.  The  parameter-estimation 
algorithm  has  attempted  to  accommodate  the  extreme 
distribution  by  increasing  the  sensitivity  parameter  value 
in  Condition  3 relative  to  Conditions  1 or  2 (see  Table  5D), 
thus  decreasing  the  error  in  predicting  that  particular 
response  distribution.  As  a consequence,  however,  the 
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prediction  of  other  response  distributions  are  now  more 
extreme  than  they  otherwise  would  be.  This  effect  can  be 
seen  in  the  predicted  distributions  for  most  other  stimuli 
in  Condition  3. 

Over-all,  however,  the  model,  using  the  perceptual 
space  derived  from  similarity  judgments  within  the  context 
of  the  full  stimulus  set,  predicts  quite  well  the  confusion 
matrices  for  partial  identification  tasks  in  which  different 
subsets  of  the  stimuli  are  identified.  The  predictions  are 
improved,  as  in  the  complete  identification  task,  when  a 
fourth  dimension  is  added  to  the  MDS-derived  perceptual 
space  . 

Adaptive  Tuning 

Adaptive  Tuning  In  Identification 

Our  application  of  the  model  to  the  complete 
identification  task  and  the  three  conditions  of  the  partial 
identification  task  suggests  that  the  observers  were 
flexible  in  their  use  of  perceptual  dimensions  to  identify 
the  stimuli,  in  that  our  estimates  of  the  relative  salience 
weights  on  the  several  dimensions  varied  from  condition  to 
condition  (see  Fig.  9).  Given  that  the  same  set  of  eight 
stimuli  was  presented  in  both  tasks,  we  may  ask  what 
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motivated  the  observers  to  adjust  the  pattern  of  salience 
weights  as  they  did  across  the  different  conditions.  While 
the  set  of  stimuli  remained  constant  across  all  conditions, 
the  subset  of  stimuli  that  we  required  the  observer  to 
identify  — the  "signals"  --  changed  from  condition  to 
condition.  Furthermore,  it  was  only  for  this  subset  of 
stimuli  that  the  observer  received  discriminative  feedback 
that  indicated  which  stimulus  had  been  presented.  We 
believe  that  the  observer  was  engaged  in  an  adaptive  tuning 
process  in  which  the  relative  weighting  of  dimensions  was 
adjusted  in  order  to  maximize  the  discr iminability  of  the 
subset  of  stimulus  patterns  to  be  identified  in  that 
condition.  This  tuning  process  probably  takes  place 
gradually,  over  many  trials,  based  on  the  feedback  given  the 
observer  regarding  the  correctness  of  identification. 

Given  that  observers  were  instructed  to  maximize 
their  percentage  of  correct  identifications,  it  seems  likely 
that  this  criterion  formed  their  basis  for  tuning.  To  test 
this  hypothesis,  we  have  determined  from  the  model,  for  each 
condition  of  the  experiment,  what  pattern  of  dimension 
salience  weights  would  maximize  the  probability  of  a correct 
signal  identification.  The  optimal  pattern  of  weights, 
assuming  an  average  value  for  the  sensitivity  parameter,  is 
plotted  by  the  dashed  line  for  each  experimental  condition 
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of  both  tasks  in  Fig.  9 . While  the  detailed  agreement  of 
the  observed  and  optimal  weight  patterns  is  not  particularly 
good,  there  is  a general  cor respondence  of  observed  and 
predicted  pattern  shape  across  conditions,  with  the 
exception  of  Condition  3 of  the  8x4  task.  The  comparison 
in  this  particular  condition  is  probably  not  meaningful 
because  of  the  large  inter-observer  differences,  as 
discussed  earlier.  In  general,  the  observed  patterns  of 
dimension  salience  weights  seem  consistent  with  the 
hypothesis  that  observers  are  tuning  their  weighting  of 
dimensions  in  order  to  maximize  the  probability  of  a correct 
ident i f ication  . 

Adaptive  Tuning  In  Judgment  of  Similarity 

The  concept  of  adaptive  tuning  may  also  provide  an 
explanation  for  the  failure  of  periodicity  to  emerge  as  a 
dimension  in  the  INDSCAL  analysis  of  the  similarity 
judgments  obtained  from  our  three  observers.  If  we  assume 
that  the  observers'  perceptual  spaces  were  the  same  in  both 
the  similar ity- judgment  and  identification  tasks,  then 
periodicity  could  fail  to  emerge  as  a dimension  in  the 
INDSCAL  analysis  if  the  observers  were  according  it  zero,  or 
nearly  zero,  weight.  This  would  be  analogous  to  our 
analysis  of  Condition  2 of  the  partial  identification  task 
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in  which  dimension  3 (contrast)  was  given  zero  weight  by  all 
observers,  although  this  dimension  presumably  was  available 
since  it  was  used  in  other  conditions.  Thus,  we  suggest 
that  periodicity  was  present  in  the  observers'  perceptual 
spaces  in  the  s im i 1 ar i ty- judgment  task,  but  given  zero 
weight  in  the  adapted  pattern  of  dimension  weights. 
Moreover,  we  believe  that  the  particular  pattern  of  weights 
resulted  from  an  adaptive  tuning  process  that  sought  to 
optimize  some  aspect  of  performance,  as  in  the 
identification  task.  Since  observers  were  instructed  to  map 
their  perceived  range  of  similarities  into  numbers  so  as  to 
use  the  entire  range  from  one  to  ten,  we  speculate  that 
observers  may  have  tuned  dimension  weights  to  obtain  the 
maximal  possible  range  of  inter- st imul us  distances  over  all 
pairs  of  stimuli.  Further  work  is  required  to  decide  this 
i ssue  . 


Discussion 


A Validation  of  MDS  Procedures  and  the  Decision  Model 


Our  approach  in  this  paper  assumes  (1)  that  a set  of 
complex  stimuli  can  be  represented  perceptually  as  a set  of 
points  in  a multidimensional  psychological  space,  (2)  that 
an  MDS  procedure  can  be  used  to  derive  the  dimensions  of 
that  space  and  the  relative  loci  of  the  stimuli  within  the 
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space,  and  (3)  that  identification  confusions  can  be 
predicted  by  a simple  decision  model  based  on  weighted 
i nt er st imul us  distances.  Our  success  in  predicting 
confusion  matrices  for  individual  observers  across  several 
experimental  conditions  provides  support  for  this  approach. 

We  believe  that  this  outcome  provides  a significant 
validation  of  the  use  of  MPS  procedures  to  reveal  perceptual 
structure.  Other  evidence  for  the  validity  of  MPS-derived 
spaces  has  relied  largely  on  the  intuitive  reasonableness  of 
the  abstracted  dimensions  and  stimulus  configurations.  The 
present  outcome  provides  much  stronger  support  in  that  the 
MDS-derived  space  is  used  to  predict  data  in  a different, 
independent  task.  To  be  sure,  the  best  prediction  of 
identification  behavior  occurred  when  the  MDS-derived  space 
for  our  three  observers  was  supplemented  by  an  additional 
dimension.  Nonetheless,  the  first  three  dimensions  revealed 
by  the  MDS  procedure  for  those  observers  were  useful  in  the 
pred iction  . 

Our  results  also  support  the  decision  model  as  a 
description  of  the  identification  process.  The  model 
accounts  well  for  the  changes  in  performance  observed  across 
different  conditions  of  the  complete  and  partial 
identification  tasks.  Of  particular  interest  is  the  fact 
that  the  model  accounts  for  these  changes  in  performance  in 
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terms  of  changes  in  the  relative  salience  of  perceptual 
dimensions  rather  than  in  terms  of  changes  in  the  structure 
of  the  perceptual  space.  In  the  model,  a given  stimulus  is 
assumed  to  have  a fixed  location  in  the  perceptual  space, 
regardless  of  changing  stimulus  context. 

The  MPS/Pec i s ion-Model  Approach  as  an  Aid  in 
Predicting  Identification  Performance 

A new,  particular  identification  task  arises 
whenever  new  signals  come  under  study.  New  signals  are 
studied  when  new  sources,  sensors,  settings,  or  displays  are 
discovered,  defined,  or  devised.  Examples  are  replete  in 
medical  diagnosis,  nondestructive  testing  of  materials  and 
equipment,  military  surveillance,  analysis  of  biological 
microstructures,  or  study  of  the  perceptually  handicapped. 
Our  experiments  enhance  the  possibility  that  the  present 
approach  can  be  used  to  gain  understanding  about,  and 
predict  behavior  in,  any  particular  identification  task. 

Gaining  understanding  about  a new  set  of  signals,  in 
this  context,  means  isolating  the  perceptual  dimensions  that 
are  useful,  and  assigning  the  proper  saliences  to  those 
dimensions.  The  practical  import  of  the  approach  described 
here  is  that  perceptual  dimensions  can  be  isolated  quite 
simply  and  quickly.  We  suspect  that  observers  can  be 
economically  trained  to  use  the  useful  dimensions,  and  to 
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use  them  approximately  with  weights  that  maximize  the 
probability  of  correct  iden t i f ic at  ions  . 

We  have  pointed  out  that  our  application  of  an  MDS 
procedure  to  our  three  test  observers  did  not  define  all  of 
the  dimensions  that  were  found  useful  in  the  identification 
task,  but  that  the  additional  dimension  in  question 
(periodicity)  did  appear  in  the  four-dimensional  MDS 
solution  yielded  by  19  other  judges,  some  of  whom  had  more 
general  experience  with  signals  of  the  sort  used  than  the 
three  observers.  In  examining  a new  identification  task, 
one  might  do  well  to  employ  a rather  large  number  of  judges, 
having  expertise  with  the  general  class  of  signals  in 
question,  and  examine  carefully  the  MDS  solutions  of  larger 
d imensional ity  . 

An  alternative  might  be  to  ask  the  judges  in  the  MDS 
task  to  rate  the  " con f usab i 1 i t y"  of  the  stimulus  pairs,  with 
the  iden t i f icat ion  task  in  their  minds,  rather  than  stimulus 
similarity.  Our  three  test  observers  might  have  yielded  the 


additional 

useful  dimension 

(periodicity! 

under 

such 

instructions 

. Though  our 

present  experiments 

were 

constr ai ned 

by  our  interest  in 

val idating  MDS 

proced  ur es 

as 

they  are  usually  applied,  MDS  tasks  undertaken  specifically 
to  analyze  new  sets  of  signals  might  possibly  be  adjusted  in 
this  manner  to  make  them  maximally  useful. 
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